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(54) DEVICE FOR MONITORING AND SUPPORTING PLANT OPERATION 

(57)Abstract: 

PROBLEM TO BE SOLVED: To facilitate to educate an 
operator by displaying operation supporting images 
obtained by synthesizing ITV images of an ITV camera 
related to VR (virtual reality) images in a VR image that 
virtually displays a plant in a three-dimensional manner. 
SOLUTION: When a virtual plant display system 4 is 
started, a main controlling part 4a makes a memory in a 
VR displaying part 4g read all three- dimensional image 
data for VR images stored in a three-dimensional virtual 
plant database 4h. Next, the part 4a makes a 
communicating part 4b establish communication 
connection with an interface agent 1 . When the agent 1 
inputs a command to the part 4b here, the part 4a 
analyzes the contents of the command. Then, various 
operation support images are displayed on a CRT display 
6 by controlling respective operations of the part 4b, a 
VR controlling part 4c, an ITV displaying part 4d, a 
message displaying part 4e and a video synthesizing part 
4f as a result of this analytical result, that is, in 
accordance with the type of the command. 
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[Claim(s)] 

[Claim 1] The plant operation monitor exchange equipment characterized by to display the 
operation exchange image which compounded the FTV image of the ITV camera relevant to this 
VR image in the VR (virtual reality) image which is plant operation monitor exchange 
equipment which supports the operating staff which displays the operation exchange image 
which shows the operational status of a plant (A) on a graphic-display means (6), and supervises 
operation of a plant, and displays a plant in three dimensions virtually. 

[Claim 2] Plant operation monitor exchange equipment according to claim 1 characterized by 
changing an ITV image according to change of VR image. 

[Claim 3] Plant operation monitor exchange equipment according to claim 1 or 2 characterized 
by providing the sound information means (4j, 7) which carries out sound information of the 
contents of the operation exchange image. 

[Claim 4] claims 1-3 characterized by displaying an operation exchange image based on the 
inference result of the interface agent (1) who reasons the operational status of a current plant 
based on the operation example of the past plant — plant operation monitor exchange equipment 
given in either. 

[Claim 5] Plant operation monitor exchange equipment according to claim 4 characterized by 
changing an operation exchange image based on the voice directions of an operating staff by 
which speech recognition was carried out by the interface agent. 
[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the plant operation monitor exchange 
equipment which grasps the operational status of a plant and supports the monitor business of an 
operating staff. 
[0002] 

[Description of the Prior Art] The conventional human interface concerning the operation 
monitor of a plant is as follows. That is, the actuation input of an operating staff is performed by 
operating a touch panel, a mouse, etc. on a display, and presentation of the operational status 
information on a plant to an operating staff is performed by displaying various process values 
on a CRT display as a numeric value or graphics. 

[0003] For example, the operating staff is shown operational status information by displaying on 
a CRT display by using as a 2D-Graphics screen the connection network of the various devices 
which constitute a plant, and carrying out the digital readout of the process value of each device 
on this 2D-Graphics screen. In the former, a majority of such 2D-Graphics screens are prepared 
for every application, and an operating staff grasps the operational status of a plant by choosing 
a 2D-Graphics screen oneself. 
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[0004] 

[Problem(s) to be Solved by the Invention] Since the operational status of a plant is displayed 
two-dimensional on a CRT display with such a conventional technique, an operating staff is 
difficult to grasp the operational status of a plant intuitively. Moreover, in order to have to grasp 
the operational status of each device by choosing a 2D-Graphics screen about the device of a 
large number which constitute a plant, the actuation for grasping the operational status of a plant 
is complicated. If the scale of a plant is enlarged, since the number of devices which constitutes 
it will also increase, in a large-scale plant, especially the load of an operating staff will become 
big. 

[0005] Moreover, since it is necessary to judge the operational status of a plant based on the 
digital readout of the various process values on a CRT display, or a 2D-Graphics display, the 
experience about the operation monitor of the knowledge and plant to the configuration and 
process of a plant is required. Moreover, since it is necessary to acquire operating instructions 
about many devices in order to carry out operation of the plant, skill is required. Therefore, in 
the former, a facility (plant training simulator) and educational curriculum of the speciality for 
training an operating staff needed to be prepared. 

[0006] In addition, these people have proposed the interface agent adapting the 
case-based-reasoning technique based on the AI technique to the existing human interface 
concerning the operation monitor of such a conventional plant in Japanese Patent Application 
No. No. 202002 [ nine to ], Japanese Patent Application No. No. 231581 [ nine to ], or Japanese 
Patent Application No. No. 263609 [ ten to ]. By using the technique of case based reasoning, 
this "interface agent detects the operation example of the past similar to the operational status of 
a current plant, makes an analogy judgment of the operational status of a current plant based on 
the operation example of this past, and provides an operating staff with the voice-told message 
for plant operation exchange. 

[0007] This invention was made in view of the trouble mentioned above, and aims at the 
following points. 

(1) Reduce the load of an operating staff further. 

(2) Make cultivation of an operating staff easy. 
[0008] 

[Means for Solving the Problem] In order to attain the above-mentioned purpose, in this 
invention, as the 1st means concerning plant operation monitor exchange equipment In the plant 
operation monitor exchange equipment which supports the operating staff which displays the 
operation exchange image which shows the operational status of a plant on a graphic display 
means, and supervises operation of a plant A means to display the operation exchange image 
which compounded the ITV image of the ITV camera relevant to this VR image in VR (virtual 
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reality) image which displays a plant in three dimensions virtually is adopted. 

[0009] Moreover, in the 1st means of the above, a means to change an ITV image accordmg to 

change of VR image is adopted as the 2nd means concerning plant operation monitor exchange 

ZmTmMBB to provide the sound information means which carries out sound information of 
the contents of the operation exchange image in the 1st or 2nd means of the above as the 3rd 
means concerning plant operation monitor exchange equipment is adopted. 
[0011] A means to display an operation exchange image in the above 1st - the 3rd one of means 
as the 4th means concerning plant operation monitor exchange equipment based on the 
inference result of the interface agent who reasons the operational status of a current plant based 
on the operation example of the past plant is adopted. 

[0012] A means to change an operation exchange image in the 4th means of the above as the 5th 
means concerning plant operation monitor exchange equipment based on the voice directions of 
an operating staff in which speech recognition was done by the interface agent is adopted. 
[0013] 

[Embodiment of the Invention] Hereafter, 1 operation gestalt of the plant operation monitor 
exchange equipment concerning this invention is explained. In addition, this operation gestalt 
makes the interface agent who mentioned above link, and is constituted. 

[0014] Drawing! is the whole plant operation monitor exchange equipment block diagram in 
this operation gestalt. For a loudspeaker and 3, as for a virtual plant display system (plant 
operation monitor exchange equipment) and 5, in this drawing, a microphone and 4 are [ an 
interface agent, and 2 and 7 / the plant whose sign A is a candidate for a monitor, and 1 / input 
devices (for example, a mouse, a keyboard, etc.) and 6 ] CRT displays (graphic display means). 
[0015] The interface agent 1 reasons the operational status of Plant A by case based reasoning 
based on the various data (plant data) in which it is acquired from Plant A and the operational 
status of Plant A is shown, and outputs various kinds of voice-told messages which support an 
operating staff to a loudspeaker 2 according to this inference result while he does speech 
recognition of the voice directions inputted from the operating staff of Plant A through the 
microphone 3 and outputs to the virtual plant display system 4. 

[0016] As a result of case based reasoning by the above-mentioned interface agent 1, the virtual 
plant display system 4 compounds the image (ITV image) of the ITV camera (industrial use 
surveillance camera) with which Plant A was equipped, and the various images (operation 
exchange image) which support an operating staff based on the operator guidance of the 
above-mentioned input unit 5, and outputs them to CRT display 6. 

[0017] This virtual plant display system 4 compounds various kinds of operation exchange 
images by combining VR (virtual reality) image generated based on the result of case based 
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reasoning by the interface agent 1, an alphabetic character message, and the ITV image acquired 
from Plant A. Moreover, in order that the virtual plant display system 4 may raise the reality of 
VR image more, it generates the sound relevant to VR image, and outputs it to a loudspeaker 7. 
In addition, an input unit 5 is for inputting the specific operator guidance to the operator 
guidance and Plant A of VR image. 

[0018] Drawing 2 is the block diagram showing the above-mentioned interface agent's 1 
functional configuration. As it ** to this drawing, the interface agent 1 consists of language 
translation system la, inference-engine lb, collating Ruhr lc, and case memory Id and data 
collection section le, If of the communications departments etc., etc. 

[0019] Language translation system la synthesizes voice from the voice data inputted from this 
inference-engine lb to the voice-told message which can recognize an operating staff, and 
outputs it to a loudspeaker 2 while changing it into the voice data [ a computer ] which can be 
read and outputting it to inference-engine lb by carrying out speech recognition of the voice 
directions inputted into the microphone 3 by the operating staff. 

[0020] When it furthermore explains in full detail, this language translation system la The 
keyword spotting speech recognition which extracts and recognizes a specific keyword from the 
voice directions which the operating staff emitted, And by memorizing the voice directions of 
an operating staff inputted in the past as voice case data (past voice case data), and carrying out 
comparison inference with the voice directions (current voice case data) into which this past 
voice case data was inputted now Speech recognition which combined the case-based-reasoning 
speech recognition which recognizes voice directions is performed. 

[0021] Data collection section le collects the operator guidance (operator guidance data) to the 
measurement data of the various devices which constitute Plant A for every predetermined 
sampling time, and the plant A of an operating staff from Plant A, and outputs it to 
inference-engine lb by making these into plant data. This plant data is information which 
consists of measurement data in a sampling time, and operator guidance data at that time, and 
shows the operational status of the plant A in a sampling time. 

[0022] Inference-engine lb reasons and judges the current operational status of a plant by 
searching the operation case data (past operation case data) of the past similar to the 
above-mentioned current operation case data based on collating Ruhr lc from case memory Id 
while making case memory Id carry out the sequential storage of the above-mentioned plant 
data as operation case data (current operation case data) in which current operational status is 
shown. 

[0023] Case memory Id, sequential are recording of the current operation case data by which a 
sequential input is carried out from above-mentioned inference-engine lb is carried out as past 
operation case data (it becomes the past data with the passage of time). Collating Ruhr lc 
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specifies the collating procedure of the current operation case data and past operation case data 
based on the technique of case based reasoning, i.e., the retrieval procedure of past operation 
case data similar to current operation case data. 

[0024] The above-mentioned inference-engine lb judges the current operational status of Plant 
A by case based reasoning based on this collating Ruhr lc. In addition, the detail of the 
generation method of the current operation case data based on inference-engine lb and the 
detail of retrieval of past operation case data are indicated by each application mentioned above. 
[0025] If of communications departments mediates the communication link with 
inference-engine lb and the above-mentioned virtual plant display system 4, and they output the 
inference result of inference-engine lb, and the speech recognition result of language translation 
system la to the virtual plant display system 4 as a command. 

[0026] Moreover, it consists of these operation gestalten so that the specific operator guidance 
to Plant A can be inputted from the virtual plant display system 4 through the interface agent 1. 
If of communications departments will output to inference-engine lb, if this operator guidance 
is received from the virtual plant display system 4. And this operator guidance is inputted into 
Plant A through inference-engine lb and data collection section le. 

[0027] Then, drawing 3 is the block diagram showing the functional configuration of the 
above-mentioned virtual plant display system 4. As shown in this drawing, the virtual plant 
display system 4 consists of Maine control-section 4a, communications department 4b, VR 
control-section 4c, 4d [ of ITV displays ], message indicator section 4e, 4f [ of image 
composition sections ], 4g [ of VR displays ], and three-dimension virtual plant database 4h, and 
direct-control section 4i, sound production section 4j, ITV composition section 4k, etc. In 
addition, sound production section 4j constitutes a sound information means with the 
above-mentioned loudspeaker 7. 

[0028] Maine control-section 4a controls each actuation of communications department 4b, VR 
control-section 4c, 4d of ITV displays, message indicator section 4e, and 4f of image 
composition sections in generalization based on the operator guidance inputted from an input 
unit 5 through various command and direct-control section 4i inputted by the interface agent 1 
through communications department 4b. 

[0029] If communications department 4b mediates the communication link with the interface 
agent 1 under control by Maine control -section 4a and the various above-mentioned commands 
are received from the interface agent 1, it will be outputted to Maine control-section 4a, and if 
the operator guidance of an input device 5 is inputted from Maine control-section 4a, it will be 
outputted to the interface agent 1. 

[0030] Direct-control section 4i receives the operator guidance of the above-mentioned input 
unit 5, and outputs it to VR control-section 4c. VR control-section 4c controls generation of VR 
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image by 4g of VR displays. This VR control-section 4c outputs modification directions of VR 
image which should be displayed on CRT display 6 based on the operator guidance of the input 
device 5 inputted from control information and direct-control section 4i inputted from 
above-mentioned Maine control -section 4a to 4g of VR displays. 

[0031] Moreover, VR control-section 4c also controls the output of the sound (sound relevant to 
VR image) to the loudspeaker 7 by sound production section 4j. Sound production section 4j 
outputs the voice-told message which tells an operating staff about the operation sound or VR 
image of a device displayed for example, into VR image having changed as a sound relevant to 
VR image to a loudspeaker 7. 

[0032] To the bottom of control by Maine control-section 4a, 4d of ITV displays chooses from 
the image of two or more ITV cameras any they are, and they output ** at 4f of image 
composition sections, and ITV composition section 4k. The key point of Plant A is equipped 
with two or more ITV cameras, and they are always photoing the operating state of the various 
devices which constitute Plant A. Message indicator section 4e changes into image information 
the message information (for example, text data) inputted from Maine control-section 4a, and 
outputs it to 4f of image composition sections. 

[0033] Three-dimension virtual plant database 4h is a database which accumulates the various 
three-dimension image data for VR images which shows the configuration of the configuration 
equipment of Plant A. VR image in this operation gestalt is a walk-through image which 
displays in three dimensions the condition of moving while an operating staff checks the 
condition of the various devices in Plant A, as a virtual space. 

[0034] A three-dimension image changes according to location change of an operating staff [ in 
/ in such a walk-through image / the virtual space of Plant A ]. Therefore, a walk-through image 
is a three-dimension image which changes continuously according to the location and view of 
an operating staff on a three-dimension virtual space. The three-dimension image data about the 
configuration of the configuration equipment of the plant A required for generation of such a 
walk-through image is accumulated in three-dimension virtual plant database 4h. 
[0035] 4g of VR displays reads three-dimension image data from three-dimension virtual plant 
database 4h, they generate VR image based on the modification directions of VR image inputted 
from VR control-section 4c, and output it to 4f of image composition sections. Moreover, 4g of 
VR displays equips the interior with ITV fusion zone 4k. This ITV fusion zone of 4g carries out 
texture mapping (dynamic texture mapping) of the above-mentioned ITV image for every frame 
to a part of VR image, and outputs it to 4f of image composition sections. 

[0036] This virtual plant display system 4 is constituted so that various VR images which 
display the operational status of the various devices which constitute the plant A other than the 
above-mentioned walk-through image according to an individual may be alternatively displayed 



6 




THIS PAGE BLANK (uspto) 



2000-259233 



on CRT display 6. 4g of VR displays generates VR images various [ these ] based on the 
three-dimension image data accumulated in three-dimension virtual plant database 4h. 
[0037] Under control by the above-mentioned Maine control-section 4a, 4f of image 
composition sections compounds the various images inputted from 4d of ITV displays, message 
indicator section 4e, and 4g of VR displays, and they output them to CRT display 6. 
[0038] Next, actuation of the plant operation monitor exchange equipment constituted in this 
way is explained in detail. In addition, about actuation of the interface agent 1, since the detail is 
indicated by each application mentioned above, below, an outline is explained. 
[0039] First, to the plant A by which continuous running is usually carried out, the interface 
agent's 1 data collection section le acquires plant data for every sampling time of predetermined 
spacing, and outputs them to inference-engine lb. After mapping inference-engine lb from data 
collection section le in this way in the metric space which shows similarity with the past 
operation case data which already stored the plant data inputted periodically in case memory Id, 
it carries out a sequential output as current operation case data at case memory Id. Consequently, 
according to operation progress of Plant A, sequential are recording of the current operation 
case data will be carried out at case memory Id as past operation case data in which the 
operational status of the past plant A is shown. 

[0040] Inference-engine lb searches whether past operation case data similar to current 
operation case data exist by searching case memory Id based on collating Ruhr lc with are 
recording of case memory Id [ such ] current operation case data. For example, a mutual 
distance of current operation case data and each past operation case data is computed, 
respectively about all the combination of 1 covering the past fixed time amount width of face or 
two or more past operation case data, and current operation case data, and it is detected as 
similar operation case data with which past operation case data with the smallest distance were 
most similar to current operation case data. 

[0041] Thus, if similar operation case data are detected, inference-engine lb will reason the 
current operational status of Plant A based on the similar operation case data concerned, and 
will generate the message data in which the operational status of Plant A is shown based on the 
inference result. For example, since current operation case data will show the abnormal 
condition of Plant A when it is a thing at the time of the abnormalities of a certain device by 
which similar operation case data constitute Plant A in the past, inference-engine lb generates 
the message data which tells an operating staff about this purport. 

[0042] Language translation system la synthesizes voice from a voice-told message based on 
this message data, and outputs it to a loudspeaker 2. Consequently, by the voice-told message, 
an operating staff can recognize that the operational status of Plant A is in an abnormal 
condition, and can take required measures. 
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[0043] On the other hand, the inference result of the operational status of the plant A by the 
above-mentioned inference-engine lb and the recognition result of voice directions of the 
operating staff in language translation system la are transmitted to the virtual plant display 
system 4 from the interface agent 1 as a command to the virtual plant display system 4. If 
inference-engine lb makes the above-mentioned command transmit to the virtual plant display 
system 4 through If of communications departments, this command will be inputted into Maine 
control-section 4a through communications department 4b of the virtual plant display system 4. 
[0044] Then, detail actuation of the above-mentioned virtual plant display system 4 is explained 
with reference to the flow chart shown in drawing 4 . 

[0045] First, when the virtual plant display system 4 starts, Maine control-section 4a makes all 
the three-dimension image data for VR images accumulated in three-dimension virtual plant 
database 4h read into the memory in 4g of VR displays (step SI). It is possible to accelerate 
generation of VR image by 4g of VR displays by this, and VR image which answers promptly 
to modification directions of VR image by the operating staff can be realized. 
[0046] When reading in the memory of such a three-dimension image for VR images is 
completed, Maine control-section 4a makes communications department 4b establish 
communication link connection with the interface agent 1 (step S2). Communications 
department 4b establishes communication link connection among If of the interface agent's 1 
communications departments according to directions of Maine control-section 4a. After this, it 
will be in a standby condition until the above-mentioned command is inputted by the interface 
agent 1 through communications department 4b or Maine control-section 4a4, i.e., this virtual 
plant display system, has the input of an actuation event from an input device 5. 
[0047] Here, if a command is inputted into communications department 4b from the interface 
agent 1 (step S3), Maine control-section 4a will analyze the contents of this command (step S4). 
And various operation exchange images are displayed on CRT display 6 by controlling each 
actuation of above-mentioned communications department 4b, VR control-section 4c, 4d of ITV 
displays, message indicator section 4e, and 4f of image composition sections according to the 
class of this analysis result, i.e., command, (step S5). 

[0048] For example, when a command is what shows the abnormalities of a specific device 
based on the inference result of inference-engine lb, Maine control-section 4a outputs the text 
data which tells the abnormalities of the device concerned to message indicator section 4e in 
step S5. Consequently, message indicator section 4e generates an alphabetic character message 
image based on text data, and is made to display it on CRT display 6 through 4f of image 
composition sections. At this time, 4f of image composition sections is based on the 
tab-control-specification information inputted from Maine control-section 4a, and they set up 
the location of the display message on CRT display 6. 
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[0049] Moreover, like the above, when a command is what shows the abnormalities of a specific 
device, Maine control-section 4a outputs the directions which indicate the ITV image of the 
abnormality device concerned by selection to 4d of ITV displays. Consequently, 4d of ITV 
displays displays the ITV image of the abnormality device concerned on CRT display 6 through 
4f of image composition sections. 

[0050] Furthermore, Maine control-section 4a directs the generation output of VR image of the 
abnormality device concerned to 4g of VR displays through VR control-section 4c. 
Consequently, 4g of VR displays generates VR image of the abnormality device concerned 
based on the three-dimension image data previously read from three-dimension virtual plant 
database 4h, and they are made to display it on CRT display 6 through 4f of image composition 
sections. VR image of this abnormality device is a VR image which carried out three-dimension 
simulation of the abnormal condition of the abnormality device concerned. 
[0051] Here, in case 4g of VR displays of this operation gestalt displays VR image on CRT 
display 6, they compound the ITV image of the abnormality device concerned inputted from 4d 
of ITV displays using ITV composition section 4k in VR image. That is, ITV composition 
section 4k compounds an ITV image in VR image by carrying out texture mapping (dynamic 
texture mapping) of the ITV image to the specific field within VR image which is a 3D 
scenography for every frame. 

[0052] When an abnormality device is a "compressor", the ITV image which shows the actual 
situation of the compressor concerned of operation is more specifically displayed on the specific 
field of the compressor virtually displayed in three dimensions on VR image. Since dynamic 
texture mapping of the ITV image of this compressor is carried out to the side face of a 
compressor per frame, it serves as an animation which changes per frame like the usual ITV 
image. 

[0053] Drawing 5 is the mimetic diagram showing an operation exchange image with such a VR 
image. As shown in this drawing, the animation of an ITV image is displayed on the specific 
field of the device displayed in three dimensions by dynamic texture mapping in VR image. 
Here, although an operating staff can change the location and view over a device by operating 
an input unit 5, since dynamic texture mapping of the ITV image is carried out on a specific 
field, when an operating staff uses a specific field as a transverse plane and confronts each other 
(right pair), an ITV image can be checked by looking the best. 

[0054] Thus, while being able to control too much feeling of devotion of the operating staff to 
imagination VR image by carrying out dynamic texture mapping of the ITV image which shows 
the actual situation of a compressor of operation, and displaying it in VR image of an 
imagination compressor, the operational status of Plant A can be grasped more easily and 
correctly. 
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[0055] In addition, when the above-mentioned commands are the modification directions of an 
operation exchange image in which speech recognition was carried out by language translation 
system la, Maine control-section 4a displays the operation exchange image according to 
modification directions on CRT display 6 in step S5. For example, if the command of the 
purport which wants to check the condition of a specific device is inputted, Maine 
control-section 4a will output the directions which indicate the ITV image of the device 
concerned by selection to 4d of ITV displays, and will display the ITV image of the device 
concerned on CRT display 6. Moreover, Maine control-section 4a displays VR image of the 
device concerned on CRT display 6 by controlling 4g of VR displays through VR 
control-section 4c. 

[0056] On the other hand, in step S6, if the actuation event concerning modification directions 
of VR image is inputted into direct-control section 4i from an input device 5, this actuation 
event will be inputted into Maine control-section 4a through VR control-section 4c from 
direct-control section 4i. Maine control -section 4a analyzes this actuation event (step S7), and 
directs the display of VR image according to the class of that analysis result, i.e., actuation event, 
to VR control-section 4c (step S8). 

[0057] VR control-section 4c outputs output directions of the sound corresponding to this VR 
image to sound production section 4j while making 4g of VR displays generate VR image 
according to directions of Maine control-section 4a. Consequently, 4g of VR displays outputs 
new VR image according to an actuation event to 4f of image composition sections, and they 
are made to display it on CRT display 6. Moreover, sound production section 4j outputs the 
sound corresponding to VR image displayed on CRT display 6 to a loudspeaker 7. 
[0058] An operating staff can move virtually on a walk-through image by operating an input 
unit 5 in the inside of Plant A. If the actuation event which directs the advance to the 
predetermined direction is inputted from an input unit 5, an operating staff can change the 
location on which oneself is based on a walk-through image. For example, when an operating 
staff arrives in front of the above-mentioned compressor in an abnormal condition, sound 
production section 4j outputs the abnormality operation sound of the compressor concerned to a 
loudspeaker 7, and makes virtual experience of an operating staff more real. 
[0059] Moreover, in the above-mentioned step S6, when the actuation event concerning 
actuation of the device displayed on the walk-through image is inputted, this actuation event is 
inputted into Maine control-section 4a through direct-control section 4i and VR control -section 
4c. Maine control-section 4a transmits the actuation event concerned to the interface agent 1 
through communications department 4b (device actuation processing: step S9). And this 
actuation event is inputted into Plant A through If of the interface agent's 1 communications 
departments, inference-engine lb, and data collection section le, and operation of the device 
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displayed on the walk-through image based on the actuation event concerned is operated. 
[0060] For example, an input of the actuation event which directs the shutdown of the 
above-mentioned compressor displayed on the walk-through image from the input unit 5 
suspends operation of the compressor concerned based on this actuation event. Or if the 
actuation event of a purport which wants to acquire the measurement data of a specific device is 
inputted from an input device 5, based on this actuation event, the numeric value of the 
measurement data of the specific device concerned will be incorporated by the virtual plant 
display system 4 through the interface agent 1, and will be displayed on CRT display 6. 
[0061] Since an operation exchange image is displayed on CRT display 6 based on the inference 
result of the operational status of the plant A by case based reasoning, and the speech 
recognition result of the voice directions by the operating staff according to this operation 
gestalt, an operating staff is made [ checking the operational status of Plant A visually with the 
operation exchange image displayed on CRT display 6 in addition to the check of the 
operational status of the acoustic-sense-plant A based on a voice-told message, or ]. Therefore, 
the check of the operational status of Plant A is easy, and it is possible to mitigate the load of an 
operating staff sharply as compared with the conventional human interface concerning the 
operation monitor of a plant. 

[0062] Moreover, since an operation exchange image is displayed on CRT display 6 with VR 
image, an ITV image, or the gestalt of a message, an operating staff can grasp the operational 
status of Plant A intuitively and easily. Moreover, since directions of an operating staff are 
inputted into the interface agent 1 by speech recognition, an operating staff can input its volition 
into the interface agent 1 easily. Therefore, since skill of an operating staff is not conventionally 
needed while it is possible to mitigate the load of an operating staff also by these, it is possible 
to aim at reduction of the cost concerning compaction and a cultivation facility of an operating 
staff of a cultivation period. 

[0063] in addition, the above-mentioned operation gestalt carries out the direct control of the 
plant A based on the voice directions of an operating staff by which speech recognition was 
carried out to the interface agent 1 — as — it is not constituted. This takes into consideration 
operating it in juxtaposition to operation of the present plant A. 

[0064] That is, when the invention in this application tends to offer the human interface 
technique of the next generation concerning operation of Plant A and it is going to apply the 
invention in this application to the actual plant A, in order for an operating staff to master the 
invention in this application reasonable, coexistence with the conventional human interface is 
needed for a fixed period. Voice directions can perform actuation of the plant which needs 
accuracy by carrying out the voice directions of the actuation of an actual plant, after an 
operating staff gets used to a voice directions input to some extent in this coexistence period. 



11 




THIS PAGE BLANK (usfto) 



* 



2000-259233 



[0065] In addition, this invention is applicable to the operation monitor of all the plants that are 

not the things for a specific plant and consisted of two or more various devices. 

[0066] 

[Effect of the Invention] According to the plant operation monitor exchange equipment 
concerning this invention, the following effectiveness is done so as explained above. 
[0067] (1) In the plant operation monitor exchange equipment which supports the operating 
staff which displays the operation exchange image which shows the operational status of a plant 
on a graphic display means as the 1st means concerning plant operation monitor exchange 
equipment, and supervises operation of a plant Since the operation exchange image which 
compounded the ITV image of the ITV camera relevant to this VR image is displayed in VR 
(virtual reality) image which displays a plant in three dimensions virtually, an operating staff 
With such an operation exchange image, it is possible to judge the operational status of a plant 
correctly and easily, and, therefore, a load can be mitigated sharply. 

[0068] (2) Moreover, when an operation exchange image is constituted only from a VR image, 
for example, we are anxious about lapsing into the situation which an operating staff is absorbed 
too much in a virtual space at VR image, and distinction with reality and a virtual space cannot 
attach easily. On the other hand, too much feeling of devotion of the operating staff to 
imagination VR image can be controlled by compounding the ITV image which shows the 
situation of an actual plant in VR image. 

[0069] (3) Since an ITV image is changed according to change of VR image, an operating staff 
can grasp easily and correctly the condition of each device which constitutes a plant in each 
location of oneself in VR image. 

[0070] (4) Since the sound information means which carries out sound information of the 
contents of the operation exchange image is provided, an operating staff can grasp the 
operational status of a plant still more nearly intuitively. 

[0071] (5) Since an operation exchange image is displayed based on the inference result of the 
interface agent who reasons the operational status of a current plant based on the operation 
example of the past plant, an exact operation exchange image is displayed to the operational 
status of a current plant. Therefore, the load of the operation monitor activity operation people's 
plant is further mitigable. 

[0072] (6) Since an operation exchange image is changed based on the voice directions of an 
operating staff by which speech recognition was carried out by the interface agent, an operation 
exchange image can be easily changed with voice directions. That is, since skill is not needed 
for actuation, while not needing time amount for cultivation of an operating staff, it is possible 
[ the directions input of an operating staff is very easy and intuitive, and ] to reduce the cost 
concerning cultivation of an operating staff. 
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[Brief Description of the Drawings] 

[Drawing 1] It is the whole 1 operation gestalt block diagram of this invention. 

[Drawing 2] It is the block diagram showing an interface agent's functional configuration in 1 

operation gestalt of this invention. 

[Drawing 3] It is the block diagram showing the functional configuration of 1 operation gestalt 
of this invention. 

[Drawing 4] It is the flow chart which shows actuation of 1 operation gestalt of this invention. 
[Drawing 5] It is the mimetic diagram showing an example of the operation exchange image in 

1 operation gestalt of this invention. 
[Description of Notations] 

A.... Plant 

1 .... Interface agent 

la .... Language conversion system 

lb .... Inference engine 

lc .... Collating Ruhr 

Id .... Case memory 

le .... Data collection section 

If .... Communications department 

2 .... Loudspeaker 

3 .... Microphone 

4 .... Virtual plant display system 
4a .... Maine control section 

4b .... Communications department 

4 C....VR control section 

4 d....ITV display 

4e .... Message indicator section 

4f .... Image composition section 

4 g....VR display 

4h .... Three-dimension virtual plant database 
4i .... Direct-control section 

4j .... Sound production section (sound information means) 

4 k....ITV composition section 

5 .... Input unit 

6 .... CRT display (graphic display means) 

7 .... Loudspeaker (sound information means) 
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